An Extension of Multi-Layer Perceptron Based on Layer-Topology

There are a lot of extensions made to the classic model of multi-layer perceptron (MLP). A notable amount of them has been designed to hasten the learning process without considering the quality of generalization. The paper proposes a new MLP extension based on exploiting topology of the input layer of the network. Experimental results show the extended model to improve upon generalization capability in certain cases. The new model requires additional computational resources to compare to the classic model, nevertheless the loss in efficiency isn’t regarded to be significant. Keywords—learning algorithm, multi-layer perceptron, topology. I.INTRODUCTION eneralization is one of the main capabilities of artificial neural networks. Generalization refers to the neural network producing reasonable outputs for inputs not encountered during training [1]. A network is said to generalize well when the input-output mapping computed by the network is correct (or nearly so) for test data never used in creating or training the network. Generalization is influenced by three factors: (i) the size of the training set, and how representative it is of the environment of interest, (ii) the architecture of the neural network, and (iii) the physical complexity of the problem at hand [1]. Let’s focus on the second factor starting up by the following example (Fig. 1). (a) (b) (c) (d) Fig. 1. Four sample patterns to demonstrate human ability to differentiate Here four bitmaps of similar proportions of black and white are depicted. It’s obvious for first two bitmaps to be differentiated by humans more easily than second two. Simply speaking, that’s because of concentration of similar pixels together. Anyway the easier differentiation process is attained through observing topological character of the patterns. The goal of the paper is to propose an extension to MLP model that observes topology of input patterns to improve upon generalization capability of a neural network. Manuscript received July 15, 2005. J. Z. is with the Department of Computer Science, University of Latvia, Riga, Latvia (e-mail: [email protected]). II.THE CLASSIC MLP WITH BACK-PROPAGATION MLP is regarded to be a reference point for neural networks, and back-propagation is the basis for training supervised neural networks. The core MLP operating and training algorithm is briefly shown in (1), (2), and (3) [1], [2]. Propagation function: